Skip to content

FEAT: Add conda packaging recipes and publish tooling for mssql-python - #720

Draft
Jahnvi Thakkar (jahnvi480) wants to merge 17 commits into
mainfrom
jahnvi/conda-onboarding
Draft

FEAT: Add conda packaging recipes and publish tooling for mssql-python#720
Jahnvi Thakkar (jahnvi480) wants to merge 17 commits into
mainfrom
jahnvi/conda-onboarding

Conversation

@jahnvi480

@jahnvi480 Jahnvi Thakkar (jahnvi480) commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

AB#47315


Summary

This pull request introduces comprehensive support for building, consolidating, validating, and publishing conda packages for the mssql-python project across all major platforms. It adds new pipeline stages and jobs for conda package handling, ensures best-effort artifact collection without blocking wheel releases, and provides both test and production paths for conda release readiness and publishing to Anaconda.org. The changes are carefully designed to keep conda processes decoupled from wheel releases, ensuring robustness and flexibility.

Conda Package Build and Consolidation:

  • Added per-platform conda package builds for Windows (x64 only), macOS (osx-64 and osx-arm64), and Linux (manylinux_2_28 for both x86_64 and aarch64), with explicit comments and conditions to skip unsupported targets (e.g., win-arm64, musllinux) in build-release-package-pipeline.yml.
  • Introduced a new ConsolidateConda pipeline stage and a reusable job template (consolidate-conda-artifacts-job.yml) to gather all per-platform conda packages into a single artifact, with best-effort warnings on missing packages and no hard failures during the build.

Release Pipeline Enhancements (Test and Production):

  • Extended both the dummy and official release pipelines with parameters and stages for conda release readiness validation and publishing, including safe test modes and production publishing with proper gating and sequencing (companion package before binding per pip warning on mssql-python and odbc dependency #706).
  • Added logic to include the Anaconda Publishing variable group only when actually publishing, to avoid unnecessary secret requirements during validation or wheel-only releases.

General Improvements:

  • Added a .gitattributes rule to enforce LF line endings for shell scripts in the scripts directory.

Adds conda recipes for mssql-python and its mssql-python-odbc companion, a local build/test-before-live harness, and a draft OneBranch conda publish stage. Azure SDK deps resolve from the lean 'microsoft' Anaconda channel under --strict-channel-priority so conda-forge's azure-core recipe (which over-declares flask/six -> celery/boto3/botocore) does not bloat the environment. See conda-forge/azure-core-feedstock#71.
Copilot AI lite review requested due to automatic review settings August 14, 2026 08:50
@github-actions github-actions Bot added the pr-size: medium Moderate update size label Aug 14, 2026
Comment thread conda/mssql-python-odbc/meta.yaml Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds initial conda packaging assets for mssql-python (and its mssql-python-odbc companion) so the project can be published to the Microsoft-owned microsoft Anaconda channel, alongside the existing PyPI wheel distribution.

Changes:

  • Adds conda recipes for mssql-python and mssql-python-odbc that repackage existing wheels (no compilation).
  • Adds a local PowerShell harness to build both recipes, create a local channel, and validate import / optional live-connect.
  • Adds a draft OneBranch stage to build/test/publish conda artifacts from signed release artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
conda/onebranch-publish-conda-stage.yml Draft OneBranch stage to build from signed artifacts, gate on smoke tests, and publish to the microsoft Anaconda channel.
conda/mssql-python/meta.yaml Conda recipe for repackaging the mssql-python wheel and depending on a version-locked mssql-python-odbc.
conda/mssql-python-odbc/meta.yaml Conda recipe for repackaging the proprietary driver wheel into a companion conda package.
conda/build_and_test_local.ps1 Local “test-before-live” harness to build, index, install, and smoke test the conda packages.
Suppressed comments (1)

conda/onebranch-publish-conda-stage.yml:133

  • Same string-vs-boolean condition issue here: quoting the template expression turns it into a string, which can cause the publish step to be skipped unexpectedly even when publishToConda is true.
          - task: PowerShell@2
            displayName: 'Publish to anaconda.org/microsoft'
            condition: and(succeeded(), eq('${{ parameters.publishToConda }}', true))

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread conda/mssql-python/meta.yaml Outdated
Comment thread conda/mssql-python-odbc/meta.yaml Outdated
Comment thread conda/build_and_test_local.ps1 Outdated
Comment thread conda/onebranch-publish-conda-stage.yml Outdated
Comment thread conda/mssql-python-odbc/meta.yaml Outdated
Comment thread conda/build_and_test_local.ps1 Outdated
Recipes now honor ARTIFACTS_PY/ARTIFACTS_ODBC to install signed wheels offline (--no-index --find-links) instead of always hitting PyPI. build_and_test_local.ps1 adds --override-channels for reproducible solves and its header no longer claims an offline/no-microsoft-channel run. Draft publish stage condition compares the boolean parameter directly (eq(param, true)) instead of a quoted string.
Comment thread conda/mssql-python-odbc/meta.yaml Fixed
- Single-source versions via MSSQL_PYTHON_VERSION/MSSQL_ODBC_VERSION env (wired from the publish-stage params) so the package version and the companion pin can't drift.

- Point mssql-python-odbc license_file at the actual ODBC Driver 18 EULA + VC++ license (was MIT-primary root LICENSE); remove the resolved TODO.

- Drop the no-op azure-identity >=1.12.0 floor (microsoft channel ships CalVer).

- Re-assert the wheel platform floor via __glibc/__osx virtual-package run constraints.

- Add conda/driver_load_probe.py + run it in the gate and local harness so we prove the native ODBC driver loads, not just the Python shim.

- Publish stage: require signed wheels (no PyPI fallback), publish companion-first with --skip-existing and a #706 pair guard, and document the required resources.pipelines declaration.
Comment thread conda/driver_load_probe.py Fixed
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

82%


📈 Total Lines Covered: 7750 out of 9429
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 75.5%
mssql_python.__init__.py: 77.6%
mssql_python.row.py: 77.6%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.4%
mssql_python.pybind.connection.connection.cpp: 84.3%
mssql_python.logging.py: 85.5%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@jahnvi480
Jahnvi Thakkar (jahnvi480) marked this pull request as draft August 18, 2026 08:57
Port the productionized conda pipeline from the ADO conda-publish-pipeline branch onto GitHub conda-onboarding, replacing the earlier draft prototype:

- conda recipes: meta.yaml + build.sh + bld.bat for mssql-python and mssql-python-odbc, vendored ODBC/VC++ EULA text, and .gitattributes (LF for shell scripts).

- OneBranchPipelines conda glue: scripts/build-conda-packages.{sh,ps1}, steps/conda-build-validate-step{,-posix}.yml, steps/conda-publish-step.yml, steps/conda-release-step.yml, jobs/consolidate-conda-artifacts-job.yml.

- Wire conda legs into build/release pipelines (buildConda params on win-64/osx/linux legs + ConsolidateConda stage), preserving GitHub-only signWindowsBinaries content.

- Remove superseded draft prototype: conda/onebranch-publish-conda-stage.yml and conda/build_and_test_local.ps1.
@github-actions github-actions Bot added pr-size: large Substantial code update and removed pr-size: medium Moderate update size labels Aug 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Suppressed comments (4)

OneBranchPipelines/scripts/build-conda-packages.sh:173

  • The verify env name only includes the Python version. On macOS this script is invoked twice on the same agent (native osx-arm64, then cross-target osx-64), so the second conda create -n verify_... can fail with an existing prefix or reuse the wrong-arch env. Include the effective subdir (native vs $CONDA_SUBDIR) in the env name to avoid collisions.
for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="

OneBranchPipelines/stages/build-linux-single-stage.yml:116

  • This apt install runs before the later apt-get update in the job, so it can fail on agents with stale apt indices (common on hosted Ubuntu). Add an apt-get update (and ideally noninteractive) before installing qemu-user-static/libc6-arm64-cross.
              # qemu-user-static: run aarch64 ELF binaries on the x86_64 host.
              # libc6-arm64-cross: the aarch64 glibc runtime (loader + libc/libm/...)
              # under /usr/aarch64-linux-gnu so qemu can resolve /lib/ld-linux-aarch64.so.1
              # for the emulated aarch64 conda build/verify (QEMU_LD_PREFIX points here).
              sudo apt-get install -y qemu-user-static libc6-arm64-cross

OneBranchPipelines/steps/conda-publish-step.yml:91

  • Installing anaconda-client from PyPI without pinning a version can make releases non-reproducible and may unexpectedly break publishing if a new release introduces behavioral changes. Consider pinning to a vetted version (or using a constraints file) so publish behavior is stable.
        $ErrorActionPreference = 'Stop'
        python -m pip install --upgrade pip
        python -m pip install anaconda-client
        # anaconda-client installs the `anaconda` console script onto PATH.
        anaconda --version

OneBranchPipelines/steps/conda-build-validate-step.yml:126

  • The error message references $links (the wheel find-links directory), but conda packages are searched under ${{ parameters.outputDir }}/bld. If this trips, the message will mislead troubleshooting.
      if (-not $built) { Write-Error "No conda packages were produced under $($links)"; exit 1 }

Comment on lines +60 to +62
home: https://github.com/microsoft/mssql-python
license: MIT
license_file: ../../LICENSE
Comment on lines +72 to +76
forgeDir="$OutputDir/miniforge"
installer="$OutputDir/$mf"
url="https://github.com/conda-forge/miniforge/releases/latest/download/$mf"
echo "Downloading $url"
curl -fL "$url" -o "$installer"
Comment on lines +80 to +84
$installer = Join-Path $OutputDir 'Miniforge3-Windows-x86_64.exe'
$forgeDir = Join-Path $OutputDir 'miniforge'
$url = 'https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Windows-x86_64.exe'
Write-Host "Downloading $url"
Invoke-WebRequest -Uri $url -OutFile $installer
…uildAll on Windows

The Windows mssql-python-odbc companion conda is now built ONCE as a Python-agnostic package in the ODBC_BuildAll stage (no python in host, wheel extracted via tar in bld.bat), mirroring the single py3-none-win_* PyPI wheel, instead of once per Python on every binding leg. The per-Python mssql-python binding legs seed that prebuilt companion into their local channel (-Package binding -DriverCondaDir) so the version-locked dependency still resolves. ConsolidateConda now also pulls the ODBC_BuildAll companion, and the #706 release/publish gates use presence-pairing (keeping strict 1:1 only for per-Python companions, c>1). macOS/Linux stay per-Python (unchanged).
P0-1: invert conda/driver_load_probe.py from a fail-OPEN denylist to a
fail-CLOSED allowlist. A repackaged native ODBC driver that fails to load now
FAILS the DB-less pre-publish gate instead of passing on any unrecognized
exception. Only a clean connect or a connection-stage diagnostic the loaded
msodbcsql driver alone can emit (ODBC branding, network provider, TLS, auth)
counts as PASS. Defer 'import mssql_python' into main() so the classifier is
unit-testable without the compiled extension. Adds tests/test_026 (22 no-DB
tests).

P0-3: gate the conda release on package METADATA, not folder names/counts.
New conda/validate_conda_release.py reads each package's authoritative
info/index.json (zstd) and validates real subdir == folder, allowed subdirs,
the full (subdir x Python) binding matrix, exact/consistent versions, and #706
binding<->companion pairing. Catches a mislabeled subdir and the 8e7f217
dropped-win-64-variant regression the count gate missed. Rewires
OneBranchPipelines/steps/conda-release-step.yml to call it. Adds tests/test_027
(11 tests incl. a real .conda round-trip).

# Unreachable endpoint (nothing listens on TCP port 1) -> the driver loads,
# attempts the socket, and fails fast at the network stage.
conn_str = "Server=127.0.0.1,1;Database=x;Uid=x;Pwd=x;Encrypt=no;TrustServerCertificate=yes;"
…P0-2)

macos-latest is an Intel Mac and the arm64 Python cannot execute there (no
reverse Rosetta), so the previous osx-arm64 leg silently built NATIVE osx-64
packages and staged them under osx-arm64 (mislabeled). Cross-build for real:

- conda/*/build.sh: when the host-env Python is not executable (non-emulated
  cross-build), extract the universal2 wheel into \ with unzip instead of
  pip -- mirrors the Windows bld.bat tar path; the arm64 slice comes from the
  universal2 wheel. Native + QEMU-emulated legs keep the pip install.
- conda/*/meta.yaml: skip_compile_pyc on macOS so conda-build does not run the
  non-runnable arm64 Python for .pyc byte-compilation (Python regenerates it).
- build-conda-packages.sh: section-7 verify auto-skips the runtime import when
  the target Python can't run on the host (osx-arm64 on Intel); the static
  arm64-slice audit is the stand-in. Native/QEMU legs still import for real.
- build-macos-single-stage.yml: add condaTargetSubdir: osx-arm64 +
  continueOnError to the arm64 leg; mark osx-64 as the native/blocking leg; add
  a BLOCKING static arm64 slice audit (lipo/otool/file) asserting the shipped
  ddbc_bindings ext and macos/arm64 dylibs really contain arm64 Mach-O. Fix the
  misleading 'Apple Silicon, native' comments.
The vendored ODBC Driver 18 links crypto/auth libs that must receive conda's
security updates instead of being frozen into the payload -- mirroring
conda-forge libpq (declares openssl + krb5, vendors neither):

- openssl # [not win]: the driver dlopen's libssl/libcrypto for TLS; it is not
  an ELF NEEDED entry so overlinking can't see it. Windows uses SChannel.
- krb5 # [linux]: libmsodbcsql NEEDs libkrb5.so.3 + libgssapi_krb5.so.2 (not
  bundled). macOS uses Kerberos.framework, Windows uses SSPI.
- vc14_runtime # [win]: msodbcsql18.dll imports VCRUNTIME140.dll but the
  vendored vcredist ships only msvcp140.dll; declare the serviced conda runtime.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (2)

OneBranchPipelines/scripts/build-conda-packages.sh:184

  • The verify env name is only based on the Python version (e.g. "verify_310"). On macOS this script is invoked twice on the same agent (osx-64 and osx-arm64) sharing the same Miniforge install/outputDir, so the second invocation will fail when it tries to create an env that already exists. Include the target subdir in the env name (and/or delete any existing env before creating it) so the two runs don’t collide.
for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="
  # -c microsoft (ahead of conda-forge) so azure-core/azure-identity/msal resolve from the
  # lean `microsoft` channel, NOT conda-forge whose azure-core recipe over-declares flask/six
  # -> celery/boto3/botocore (~9 MB); see conda-forge/azure-core-feedstock#71.
  # --strict-channel-priority keeps the freshly built local companion + binding authoritative.
  "$conda" create -y -n "$envName" -c "$bld" -c microsoft -c conda-forge --strict-channel-priority --override-channels "python=$py" mssql-python

conda/validate_conda_release.py:67

  • read_index_json() uses next(...) to locate the info-*.tar.zst member inside a .conda. If the archive is malformed (missing that member), this will raise StopIteration and produce a stack trace rather than a clear validation failure message. Handle the empty case and raise a ValueError with a helpful message instead.
        with zipfile.ZipFile(path) as zf:
            info_name = next(
                n for n in zf.namelist() if n.startswith("info-") and n.endswith(".tar.zst")
            )
            info_blob = zf.read(info_name)

libodbcinst.so.2 has NEEDED libltdl.so.7 but no RUNPATH, so a minimal glibc

Linux base throws 'OSError: libltdl.so.7: cannot open shared object file' on

import. macOS already vendors libltdl.7.dylib; Linux was the inconsistent

outlier. Two parts:

1) eng/scripts/patch-linux-odbc-libs.sh (new): maintainer/CI tool, run in a

   manylinux_2_28 container, that sources the glibc libltdl.so.7, copies it

   next to libodbcinst.so.2, and patchelf --set-rpath '\' so the driver

   resolves it from its own dir. Skips Alpine/musl by design.

2) build-odbc-all-stage.yml: the wheel content verifier now REQUIRES

   libltdl.so.7 in both manylinux_2_28 payloads (fail-closed). This gate stays

   red until a maintainer runs the patch script and commits the produced

   libltdl.so.7 + rpath-patched libodbcinst.so.2 under

   mssql_python_odbc/libs/linux/debian_ubuntu/<arch>/lib/ (cannot be produced

   on the Windows-only odbc build host).
Decision 2(b): Alpine is a supported, PR-tested platform, so its wheels must be

fixed too (not dropped, not documented-as-limitation). Extends the libltdl

self-contained-payload work to musl:

- eng/scripts/patch-linux-odbc-libs.sh now auto-detects libc + arch and patches

  the matching distro subtrees: a glibc-built libltdl (manylinux, dnf) for

  debian_ubuntu/rhel/suse, a musl-built libltdl (Alpine, apk add libtool) for

  alpine. One build per (libc, arch) serves all that arch's distro subtrees. It

  also drops a per-dir LIBLTDL_LGPL_LICENSE.txt notice.

- mssql_python_odbc/libs/LICENSING: document libltdl (GNU Libtool, LGPL-2.1-or-

  later, dynamically linked) covering the existing macOS libltdl.7.dylib and the

  new Linux libltdl.so.7 (compliance; precedent = macOS already vendors it).

- build-odbc-all-stage.yml: musllinux_1_2_{x86_64,aarch64} wheels now also REQUIRE

  libltdl.so.7 (fail-closed), since the Alpine test leg's system libltdl masks

  the OSError today.

- eng/scripts/audit_bundled_binaries.py (new): allowlist-driven ELF/Mach-O/PE

  dependency audit (gate step 3). Every dep must be BUNDLED, BASE (OS/libc), or

  DECLARED (openssl/krb5 [linux], vc14_runtime [win]); anything else fails. ELF

  binaries that need a bundled sibling must carry an \ RUNPATH; macOS

  absolute non-system install names (e.g. /opt/homebrew) fail regardless of

  basename; --require-arch asserts Mach-O slices (the Intel-agent substitute for

  the arm64 runtime import). Validated locally: Windows PASS, Linux FAIL on the

  missing libltdl.so.7, macOS FAIL on libodbc.2.dylib's /opt/homebrew libltdl.
…dmine)

libodbc.2.dylib is the unixODBC driver MANAGER; mssql-python loads the driver
(libmsodbcsql) DIRECTLY and never goes through it. The binary-dependency audit
proves it is a pure unused leaf on BOTH arches -- nothing in the payload depends
on it, while the active chain libmsodbcsql -> @loader_path/libodbcinst.2.dylib ->
@loader_path/libltdl.7.dylib is fully self-contained. libodbc.2.dylib is also the
ONLY macOS binary carrying an absolute build-host Homebrew path (/opt/homebrew on
arm64, /usr/local on x86_64 -> libtool/lib/libltdl.7.dylib) that does not exist on
an end-user Mac -- a latent landmine. Deleting the committed binary removes it from
BOTH the PyPI wheel (setup_odbc packages the committed libs tree) and conda
(build.sh repackages the wheel) -- one source. After the drop the macOS audit is
fully self-contained (every dep BASE/BUNDLED).

Also: the audit now SKIPS wrong-format non-library files (the Windows PE .rll
localized-message resources the driver ships in every OS payload) instead of
erroring on them -- only a wrong-format file that NAMES itself a shared library
(incl. versioned sonames) is a real cross-platform leak. And drop the now-stale
libodbc.2.dylib overlinking justification from the conda recipe comment
(error_overlinking stays off for the broader signed-repackage reason).
Revert the libodbc.2.dylib deletion from d6e790c -- that binary is part of the
well-tested macOS payload already shipping on PyPI, so it stays untouched. Restore
it on BOTH arches.

Instead of removing a shipped file to satisfy the audit, fix the audit's model to
match reality: only defects on the transitive load graph rooted at the DRIVER the
binding actually dlopens (libmsodbcsql / msodbcsql, plus msodbcdiag + mssql-auth
on Windows) can affect a real load and gate the build. A vendored library that
nothing in that graph pulls in -- e.g. the unixODBC driver-manager libodbc.2.dylib
on macOS, which mssql-python never loads (it opens libmsodbcsql directly) -- is
inert dead weight; a hardcoded install-name / rpath / missing arch slice on it can
never break a load, so it is reported UNREACHABLE (informational), not VIOLATION.

- audit_bundled_binaries.py: add _DRIVER_ROOT_RE + _reachable_from_driver (BFS over
  same-payload dep basenames from the driver root; if no root is found, fail closed
  by treating everything reachable). VIOLATION / RPATH / missing-arch-slice findings
  on unreachable binaries downgrade to UNREACHABLE and do NOT fail the gate.

Validated: macOS arm64 + x86_64 PASS (libodbc.2.dylib's /opt/homebrew|/usr/local
libtool path now UNREACHABLE; the real chain libmsodbcsql -> libodbcinst.2.dylib ->
libltdl.7.dylib stays enforced). Windows x64 PASS. Linux still FAILS on the REACHABLE
libodbcinst.so.2 -> libltdl.so.7 gap (correct fail-closed, pending the committed
patched payload). black + py_compile clean.
…RUNPATH

The vendored Linux libmsodbcsql ships RUNPATH=$ORIGIN only. Its DECLARED,
NOT-bundled load-time deps (libkrb5.so.3, libgssapi_krb5.so.2) and the
libssl/libcrypto it dlopens lazily at Encrypt=yes therefore resolve only from the
binary's own dir. In a conda env those live in <PREFIX>/lib, which conda does not
put on the loader path (no LD_LIBRARY_PATH; binary_relocation=false injects no
rpath), so they are UNREACHABLE -- masked today only by a system copy.

- patch-linux-odbc-libs.sh: also set the driver RUNPATH to
  '$ORIGIN:$ORIGIN/../..(x8)' so it keeps finding co-located libodbcinst AND
  reaches the conda <PREFIX>/lib. glibc consults the caller's DT_RUNPATH for both
  its own direct NEEDED and its own dlopen(soname), so one relative climb fixes
  both. Inert for the PyPI wheel (venv lib empty -> loader falls through to the
  system), so one committed binary serves both.
- audit_bundled_binaries.py: add a masking-immune static check -- a reachable ELF
  with a DECLARED NEEDED dep must carry a RUNPATH that climbs above $ORIGIN, else
  fail. Also fix the reachability model to resolve a dep to a SAME-DIRECTORY
  sibling (matching $ORIGIN/@loader_path/same-dir DLL), so the 7 side-by-side
  driver stacks no longer collapse across identical sonames -- which was masking
  the glibc libodbcinst -> libltdl.so.7 defect as UNREACHABLE.
… is reachable

The Linux libmsodbcsql resolves its OpenSSL backend (libssl/libcrypto) by dlopen
at TLS time -- there is no libssl/libcrypto DT_NEEDED or soname in the binary, so
an Encrypt=no connect never touches it. The existing DB-less driver_load_probe.py
(Encrypt=no) therefore CANNOT reveal an unreachable OpenSSL, e.g. a conda env
whose declared openssl in <PREFIX>/lib the driver's RUNPATH does not reach. Only
a real Encrypt=yes handshake forces the dlopen.

- conda/tls_connect_probe.py: fail-closed live gate. Forces Encrypt=yes (so the
  pre-login TLS handshake is mandatory) and classifies the outcome -- reaching
  authentication / database (clean connect, Login failed / 18456, Cannot open
  database) is positive proof OpenSSL loaded and negotiated; an SSL Provider /
  libssl/libcrypto / dlopen error fails closed as OPENSSL BACKEND UNREACHABLE.
  Reads CONDA_TLS_PROBE_CONN; SKIPS loudly (never silently passes) when unset.
- tests/test_028_tls_connect_probe.py: no-DB unit tests for the classifier and
  force_tls (append/override/idempotent).
- build-conda-packages.sh: run the gate (BLOCKING) after the DB-less probe only
  when CONDA_TLS_PROBE_CONN is set, else skip loudly.

CAVEAT documented in the probe/script: this end-to-end gate is conclusive ONLY on
a minimal base with no system OpenSSL (a system libssl masks an unreachable conda
copy, exactly as full CI agents do). The masking-IMMUNE guard remains
eng/scripts/audit_bundled_binaries.py, which requires an $ORIGIN/.. RUNPATH climb
by reading the binary regardless of any system libs.

def test_force_tls_appends_when_absent():
probe = _load_probe()
out = probe.force_tls("Server=localhost;Database=x;Uid=x;Pwd=x;")

def test_force_tls_overrides_encrypt_no():
probe = _load_probe()
out = probe.force_tls("Server=localhost;Encrypt=no;Database=x")

def test_force_tls_is_idempotent():
probe = _load_probe()
once = probe.force_tls("Server=localhost;Database=x")
…& musl)

Completes the vendored Linux ODBC payload by running eng/scripts/patch-linux-odbc-libs.sh in manylinux_2_28 (glibc) and musllinux_1_2 (musl) containers for x86_64 and arm64. Per distro/arch subtree: adds a matching-libc libltdl.so.7 + LGPL notice beside libodbcinst.so.2; sets RUNPATH=$ORIGIN on libodbcinst.so.2 so its bundled libltdl resolves on a minimal base (fixes the OSError: libltdl.so.7 import failure); and sets RUNPATH=$ORIGIN:$ORIGIN/../..(x8) on libmsodbcsql so the conda-declared krb5/openssl in <PREFIX>/lib resolve. Verified with audit_bundled_binaries.py (all deps BUNDLED/BASE/DECLARED, no VIOLATION/RPATH finding across all 7 stacks) and a ctypes load of libodbcinst.so.2 on a libltdl-free glibc base.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: large Substantial code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants